Manhattan World

This paper argues that many visual scenes are based on a ”Manhattan” three-dimensional grid which imposes regularities on the image statistics. We construct a Bayesian model which implements this assumption and estimates the viewer orientation relative to the Manhattan grid. For many images, these estimates are good approximations to the viewer orientation (as estimated manually by the authors). These estimates also make it easy to detect outlier structures which are unaligned to the grid. To determine the applicability of the Manhattan world model we implement a null hypothesis model which assumes that the image statistics are independent of any three dimensional scene structure. We then use the log-likelihood ratio test to determine whether an image satisfies the Manhattan world assumption. Our results show that if an image is estimated to be Manhattan then the Bayesian model’s estimates of viewer direction are almost always accurate (according to our manual estimates), and vice versa.